Search for: All records

An increasing number of studies apply tools powered by large language models (LLMs) to interview and conversation-based research, one of the most commonly used research methods in CSCW. This panel invites the CSCW community to critically debate the role of LLMs in reshaping interview-based methods. We aim to explore how these tools might (1) address persistent challenges in conversation-based research, such as limited scalability and participant engagement, (2) introduce novel methodological possibilities, and (3) surface additional practical, technical, and ethical concerns. The panel discussion will be grounded on the panelists’ prior experience applying LLMs to their own interview and conversation-based research. We ask whether LLMs offer unique advantages to enhance interview research, beyond automating certain aspects of the research process. Through this discussion, we encourage researchers to reflect on how applying LLM tools may require rethinking research design, conversational protocols, and ethical practices. 

Producing open-ended creative work through crowdsourcing remains a challenge, as workers often lack domain expertise, and requesters struggle to provide scalable guidance. Can the workers themselves create materials that guide subsequent workers? In this paper, we prototype a workflow for emergent scaffolding, where hints, rubrics, and examples are generated by crowd workers after attempting the task. We demonstrate how an iterative Train-Try-Reflect-Synthesize pattern—supported by LLMs—can produce a structured rubric with graded examples to guide subsequent workers on a task to create digital illustrations for scientific papers. To evaluate this strategy, we conducted a between-subjects experiment with three conditions: baseline instructions, generic examples, and emergent scaffolding. Participants in the emergent scaffolding condition created significantly better illustrations, as rated by blind-to-condition judges, compared to generic examples or instructions only. While self-efficacy ratings were mixed across conditions, emergent scaffolding participants provided better feedback during their post-task reflections. We discuss the potential for emergent scaffolding to support and scale up complex, creative tasks in crowdwork. 

Advances in speech technologies and generative AI (Gen AI) have enabled the possibility of generating conversational cues to improve engagement and creativity during small-group discussions. Such cues could contextually adapt and guide a live conversation, or conversely, serve as a distraction. How do conversational cues impact ideation and social interaction? How does the meeting modality impact the effectiveness of cues? We built a system, CueTip, to generate and deliver real-time contextual conversational cues using the GPT 4o-mini model. In a 2x2 study, N=172 participants in dyads completed a brainstorming task where they received cues or not during either a virtual or in-person discussion. Cued participants’ ideas and discussions were more topically diverse than Non-Cued participants. In-person groups noticed slightly more cues on average than virtual groups. We discuss implications for designing effective conversational cues. 

Voice-based social media platforms that enable attendees to have real-time, ephemeral interactions with each other—such as X-Spaces, Discord, and Clubhouse—have seen considerable growth in recent years. While prior research on these spaces has predominantly focused on moderating harms, our work seeks to understand emergent practices employed by hosts to proactively shape their discussion space— focusing on the facilitation aspect of moderation duties. Drawing on facilitation strategies, we study these practices through three comprehensive studies using mixed-methods: survey of social-audio users, co-design interviews, and analyzing training sessions for hosts. Our findings reveal insights into the issues faced by hosts and attendees, current facilitation practices, opinions on technological solutions, and factors that could be responsible for some of the identified issues such as the available training for hosts. We found that hosts themselves are often significant sources of issues due to practices such as focusing more on self-promotion than facilitating discussions. In addition, host training sessions seem to encourage behaviors that contribute to the negative perception of hosts. We draw on outcomes from co-design interviews to guide the design of future tools to support hosts in facilitating social-audio spaces. Our findings provide insights that could help create a more positive experience for both hosts and attendees. 

Generative AI has enabled novice designers to quickly create professional-looking visual representations for product concepts. However, novices have limited domain knowledge that could constrain their ability to write prompts that effectively explore a product design space. To understand how experts explore and communicate about design spaces, we conducted a formative study with 12 experienced product designers and found that experts — and their less-versed clients — often use visual references to guide co-design discussions rather than written descriptions. These insights inspired DesignWeaver, an interface that helps novices generate prompts for a text-to-image model by surfacing key product design dimensions from generated images into a palette for quick selection. In a study with 52 novices, DesignWeaver enabled participants to craft longer prompts with more domain-specific vocabularies, resulting in more diverse, innovative product designs. However, the nuanced prompts heightened participants’ expectations beyond what current text-to-image models could deliver. We discuss implications for AI-based product design support tools. 

An increasing number of tools now integrate AI support, extending the ability of users—especially novices—to produce creative work. While AI could play various roles within such tools, less is known about how the positioning of AI affects an individual’s cognitive processes and sense of agency. To examine this relationship, we built a collaborative whiteboard plugin that integrates an LLM into design templates to facilitate reflective brainstorming activities. We conducted a between-subjects experiment with N=47 participants assigned to one of three versions of AI-support—No-AI, AI input provided incrementally (Co-led) and AI provided all at once (AI-led)—to compare the allocation of cognitive resources. Results show that the positioning of AI scaffolds shifts the underlying cognition: AI-led participants devoted more time to comprehension and synthesis, which yielded more topically diverse problems and solutions. No-AI and Co-led participants spent more time revising content and reported higher confidence in their process. 

Discussing data related to news is one form of participatory journalism. \textit{Data talk} can promote data literacy, foster community around grounded understanding, and encourage civic engagement. What are the challenges and potential of scaffolding data talk on news websites? To understand current practices and perceptions around future visions for audience engagement in news, we interviewed 12 diverse stakeholders including journalists, data scientists, readers, and moderators. To provoke future thinking, we asked participants to react to our interactive design probe with three scenarios: questioning data on a chart, adding data views, and telling personal data stories. We identified several challenges stakeholders face with data talk, including the fast news cycle hindering thorough data discussions and the difficulty of creating accessible insights and visuals. Despite significant data wrangling, discussions around the data remain rare. Participants reacted positively to the interactive scaffolding features in the three design scenarios, noting that these features can make data an effective entry point for discussions, scaffold audience participation in the data pipeline, and lower the barrier for engagement. 

While creativity is often romanticized as a serendipitous ’aha’ moment of insight, in reality, it is an iterative process that often involves searching for information on the Web. In this paper, we investigate the role of web search throughout the creative process. We conducted a longitudinal study involving 15 professionals engaged in creative work, such as scientific research, startup product design, and policy development, observing them throughout their one to six-month-long projects. We developed Web ChronoLogger, a browser extension that logs Web Search and Project document activity over the course of the project in an intuitive, transparent, and privacy-preserving manner. Additionally, we collect qualitative insights from participants reflecting on their logs through weekly surveys and a post-study interview. We find quantitative patterns in how participants search the web and work with information in working documents throughout their creative projects. Web search was used even when generating ideas and defining goals, stages often assumed to involve just mental processes. Further, patterns in the content, structure, and edit history of how participants work with information found on the web can encode signals about the user’s context, such as patterns and gaps in their knowledge, project goals and progress, and work style. This study’s longitudinal perspective provides a foundation for building the future of web search tools in ways that support the entire creative workflow.